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[57] ABSTRACT 

A focus detecting method includes the step of project- 
ing the real image of an observation object including a 
plurality of object patterns onto an image pickup device 
through an optical system and producing image data 
from an output of the image pickup device, the step of 
calculating correlation values of the image data of each 
of the plurality of object patterns and the image data of 
a prestored reference pattern while varying the relative 
positional relation among the image pickup device, the 
optical system and the observation object in the direc- 
tion of the optical axis of the optical system, and the step 
of judging a relative positional relation giving the maxi- 
mum correlation value as an in-focus state. An appara- 
tus is provided for carrying out the above-described 
focus detecting method. 

9 Claims, 9 Drawing Sheets 
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of the accompanying drawings, the real image of an 

PATTER CORRELATION TYPE FOCUS observation object 9 including a plurality of obiect 

DETECTING METHOD AND FOCUS DETECTING patterns is projected onto an image pickup device 11 

APPARATUS through an optical system 10, image data obtained from 

RArTfrpnnwn nn xutj ixrvrwri^ 5 ^ image pickup device 11 is introduced (into an image 

BACKGROUND OF THE INVENTION memory 2), correlation values of the image data of each 

1. Held of the Invention of the plurality of object patterns and the image data of 
This invention relates to a focus detecting method a reference pattern prestored (in an external store 3) are 

and a focus detecting apparatus suitable for application, calculated while at least one of the image pickup device 
for example, to the auto focus mechanism portion of a *° 11, the optical system 10 and the observation object 9 is 

semiconductor inspecting apparatus for recognizing varied in the direction of the optical axis of the optical 

and inspecting a predetermined pattern from among system 10 and the relative positional relation therebe- 

objects to be inspected or a video camera. tween is varied, and a relative positional relation which 

2. Related Background Art gives the maximum correlation value is judged as an 
A semiconductor inspecting apparatus for recogniz- 15 in-focus state. 

ing and inspecting a predetermined pattern from among In this case, design may be made such that when the 

objects to be inspected or a video camera used for com- image pickup device 11, the optical system 10 and the 

mon people's livelihood is provided with an auto focus observation object 9 are in a certain relative positional 

mechanism for grasping the clear-cut real image of an relation, the correlation value of the image data of each 

image pickup object always on an image pickup ele- 20 of ^ luraHty of ofeject ^ ^ ^ ^ f 

ment. As automatic focus detecting methods for an ^ reference pattern is found and an object pattern of 

image pickup object, there are usually used an active which « value fc a pred etermined value or 

method of applymg lUummatmg light or laser light for ^Icr is selected from among the plurality of object 

focal position detection to the image pickup object and !,h^c ™a a„+w t u a i uojcci 

monitoring any variation in the intensity of reflected 25 S^ASS^hJ^ E 
light to thereby effect focusing, and a passive method of 5™2 ^SZZ^^P W system 

delecting the signal of the edge portion of any pattern %L£^^ "^t ^ ^ 

from a picked-up image signal and controlling the focal \ T * P OS, * on ? t0 /* el * tWQ P osl " 

position so that the image may become sharpest *° nal T ff°? dlfferm S therefrom, and the correlation 

The active method is often used, for example, for 30 value of the image data of said selected object pattern 

microscopic observation images or the like. The passive ™ the image data of said reference pattern is found, 

method is often used, for example, for the automatic . Deffl S? ™ y ^ be made such that w hen the real 

focusing or the like of video cameras. vnagp of the observation object is to be displayed on a 

However, in case of the active method, the detection screen ( a CRT display 5) on the basis of the image data 

object of the focal position is limited only to the image 35 obtained from the image pickup device 11, at least one 

pickup object surface in a predetermined focal position ? f ^ r ? al image of the observation object 9 and the 

detection area. Therefore, where the position of the *aiage pickup device 11 is relatively moved in a direc- 

image pickup object gradually deviates in the image tion perpendicular to the optical axis of the optical 

field, there is the possibility of focusing being effected system 10 so that one of the plurality of object patterns 

on a different image pickup object surface. On the other 40 of which the correlation with the reference pattern is 
hand, where the passive method is used, if a plurality of highest may come to a predetermined position on the 

objects differing in distance exist in the image field, image field. 

there is the possibility of focusing being effected on an Also, the focus detecting apparatus according to the 

object not intended by the operator, depending on present invention, as shown, for example, in FIG. 1, has 

which object is selected as the object of edge detection. 45 311 image pickup device 11 for picking up the real image 

That is, the focus detecting methods according to the of 311 observation object 9 including a plurality of object 

prior art have suffered from the inconvenience that patterns through an optical system 10, drive means 13 

focusing can be effected only on a particular pattern or *° r driving at least one of the image pickup device 11, 

a particular object irrespective of the position thereof in the optical system 10 and the observation object 9 in the 

the image field. Even in the focus detecting methods 50 direction of the optical axis of the optical system, first 

according to the prior art, if, for example, the operator memory means 2 for storing therein the image data of 

changes the focus detection area in the image field in the object patterns obtained from the image pickup 

pursuit of the movement of an image pickup object, device 11, second memory means 3 prestoring the 

focusing can be effected on that image pickup object, image data of a reference pattern therein, correlation 

but this not only requires a long processing time but also 55 value calculating means for calculating correlation val- 

is bad in operability. ues of the image data in the first memory means 2 and 

SUMMARY OF THE INVENTION IZtnl^ memory means 3 while 

controlling the drive means 13 to thereby vary the rela- 

In view of the above-noted point, the present inven- tive positional relation among the image pickup device 

tion has as an object thereof the provision of a focus 60 11, the optical system 10 and the observation object 9, 

detecting method which can accomplish focus detec- and judging means 1 for judging a relative positional 

tion to a particular pattern or a particular object irre- relation which gives a maximum value to said correla- 

spective of the position thereof in the image field. tion value as an in-focus state. 

The present invention has as a further object thereof According to such focus detecting method of the 
the provision of a focus detecting apparatus directly 65 present invention, the image data of the reference pat- 
usable to carry out such a focus detecting method. tern which is the reference of focus detection is pre- 

The focus detecting method according to the present stored (registered). For example, pattern matching is 

invention is such that as shown, for example, in FIG. 1 effected to recognize an object pattern identical or simi- 
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lar to the reference pattern from among the images of point evaluation function, depending on the manner of 
the observation object including the plurality of object setting the threshold value levels, 
patterns. When the degree of coincidence between the In contrast, in the present invention, for example, for 
reference pattern and the object patterns is made into a the image data of FIG. 8A of the accompanying draw- 
numerical correlation value of pattern, the correlation 5 ings, the normalization of gradation is effected so that as 
values of pattern at positions whereat object patterns shown in FIG. 8B of the accompanying drawings, the 
having high correlation with the reference pattern exist difference between the maximum value and the mini- 
(hereinafter referred to as candidate points) exhibit dif- mum value of the image data may become a predeter- 
ferent curves for a variation in the focal position. In this mined value (e.g. a maximum difference value) It has 
case, if the relative positional relation among the image 10 been found that according to this, a substantially con- 
pickup device and the optical system and the observa- stant funcdon which a t ^ at ^ focal 
tion object is gradually changed m the direction of the po int as shown in FIG. 9 of the accompanying drawings 
optical axis of the optical system, when the focus on an is obtained ^ the focal point evaluation func- 
object pattern identical or similar to the reference pat- tioiL Accordingly> when ^ cu ^ t relation value 
hT,^ TF 5 °\ th *\obszrv*uon object is 15 0 f pattern is found, the difference in the focal position to 
detected the correlation value of pattern becomes max- the foca] ^ ^ be wnsiderabl l cc J££ 
imum and therefore, focus detection can be accurately ^ focus ^ e tection can be effected at a higher 22? 

Also, if design is made such that after ZSte 

Also, when the focal position changes and the image 20 "I!^ I ^ ren ? ^ T 

of the matching object pattern becomes a blurred im- S^ d \° ne * Cmdl ^ p ° mtS whlc * has the 

age, the correlation value of pattern becomes gradually ht s£* ' omla * on comes to a predetermined position 

lower and the relation of the focal position and the ° n ^ff™ ^f*' foc ^ sm S and alignment can 

correlation value of pattern to the object pattern identi- b . e at a ? me f ° r *? object pattern ldentical <>r 
cal or similar to the reference pattern becomes a ranc- 25 Sim ^ ar to registered reference pattern, 

tion which assumes a maximum value at the focal point . Mso > acc ° rdm S to *e focus detecting apparatus of 

as shown, for example, in FIG. 3 of the accompanying ! he Presentmventioii, the above-described focus detect- 

drawings (hereinafter referred to as the 'focal point mg meth r od can 1x5 ^tactly carried out as an alignment 

evaluation function"). On the other hand, for a pattern method for a Particular pattern, 
differing from the registered reference pattern, even if 30 BRIEF DESCRIPTION OF THE DRAWINGS 
the focal position is changed, there will not be obtained 

the focal point evaluation function as shown in FIG. 3 . FIG ' 1 ^ a block digram showing the basic construc- 
which assumes a high peak at the focal point. Likewise, tion of a ^ Irst embodiment of the present invention, 
when there is no pattern identical or similar to the regis- *FIG. 2 is a diagram used for the illustration when the 
tered reference pattern, the correlation value of pattern 35 ^ of templates are registered in the first em- 
assumes a very low level and in any case, focus detec- bodiment. 

tion is not effected. f 10 * 3 is a graph showing an example of the focal 

Accordingly, the focal position is changed toward point evaluation function used in the first embodiment, 

the peak of the correlation value of pattern in accor- FKj. 4 ^ a flow cn art showing the focusing operation 

dance with the focal point evaluation function of FIG. 40 °^ tne first embodiment 

3 obtained for only the object pattern identical or simi- FIGS. 5A and 5B are graphs used for the illustration 
lar to the registered reference pattern, whereby the ? f candidate points detected in the process of the focus- 
focusing operation is executed. m S operation of the first embodiment 

Description will now be made of the operational FIGS. 6A, 6B and 6C are diagrams used for the illus- 

effect when the normalization of the gradation of the 45 tration when the image data of templates of different 

introduced image data is effected. Generally, it is neces- heights are registered in the first embodiment, 

sary that the focal point evaluation function be of a FIG. 7 is a graph showing two focal point evaluation 

shape which exhibits a great peak at a correct focal functions obtained by the use of the two templates of 

point, as shown in FIG. 3. For that purpose, however, FIGS. 6A to 6C. 

a predetermined appropriate preprocess is sometimes 50 FIGS. 8A and 8B are graphs used for the illustration 

necessary during the preparation of the image data of of the gradation normalizing operation of a second 

the reference pattern and during the introduction of the embodiment of the present invention. 

image data of the observation object For example, the FIG. 9 is a graph showing a normalizing correlation 

original image data of the observation object is multi- search model curve used in the second embodiment. 

value gradation, and when binary images are compared 55 FIGS. 10A, 10B and 10C are graphs used for the 

with each other during comparison, it is necessary that illustration when image data is binarized to thereby find 

the image data of the multivalue gradation be binarized. a focal point evaluation function. 

For the binarization, it is necessary to set a threshold FIG. 11 is a diagram showing a plurality of templates 

value level. However, when, for example, for the image in a third embodiment of the present invention. 

data of FIG. 10A of the accompanying drawings, three 60 FIG. 12 is a graph showing focal point evaluation 

threshold value levels thl-th3 are set as shown in FIG. functions corresponding to the plurality of templates in 

10B of the accompanying drawings, the focal point the third embodiment. 

evaluation function for the threshold value level th2 FIGS. 13A, 13B, 13C and 13D are diagrams showing 

becomes as indicated by a function 22 in FIG. 10C of examples of standard templates in a fourth embodiment 

the accompanying drawings, and the focal point evalua- 65 of the present invention. 

tion function for the threshold value level thl or th3 FIG. 14 is a diagram showing the observation screen 

becomes as indicated by a function 23 in FIG. 10C and of a sample of which the focus detection is effected in 

thus, there is sometimes obtained an unsuitable focal the fourth embodiment 
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DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Some embodiments of the present invention will 
hereinafter be described with reference to the drawings. 
A first embodiment is one in which the present inven- 
tion is applied to the automatic focus detecting appara- 
tus of an image processing apparatus for recognizing a 
predetermined pattern from among images in an image 
field. 

First Embodiment 

Referring to FIG. 1 which shows the basic construc- 
tion of the hardware of the first embodiment, the refer- 
ence numeral 1 designates a central processing unit 
(hereinafter abbreviated as "CPU") for controlling the 
operation of the entire apparatus, the reference numeral 
2 denotes an image memory comprising a random ac- 
cess memory (RAM), and the reference numeral 3 des- 
ignates an external store comprising, for example, a 
hard disk device. The reference numeral 4 denotes a 
digital/analog (D/A) converter, and the reference nu- 
meral 5 designates an image display. The image data in 
the image memory 2 is supplied to the image display 5 
through the D/A converter 4. Thereby, an image corre- 
sponding to the image data stored in the image memory 
2 is intactly displayed on the image display 5. 

The CPU 1 writes, for example, the image data of a 
cursor or the like into the image memory 2 and also 
causes the external store 3 to store therein the image 
data of a reference pattern (hereinafter referred to as the 
"template") which provides the reference of focus de- 
tection. The CPU 1 further calculates the correlation 
between the image data read out from the image mem- 
ory 2 and the image data read out from the external 
store 3, and effects focus detection which will be de- 
scribed later in conformity with the result of the calcu- 
lation. The reference numeral 6 designates a CRT dis- 
play for the CPU 1, and the reference numerals 7 and 8 
denote a keyboard and a simple coordinates input unit 
(for example, a mouse or the like), respectively, which 
are connected to the CPU I. The operator operates the 
keyboard 7 and the coordinates input unit 8 to thereby 
make the CPU 1 execute a desired operation while 
watching the CRT display 6. 

The reference numeral 9 designates a sample (an 
observation object), the reference numeral 10 denotes 
an optical system for observation, and the reference 
numeral 11 designates an image pickup element such as 
a charge coupled type image pickup device (CCD). The 
real image of the sample 9 at a predetermined magnifi- 
cation is formed on the image pickup surface of the 
image pickup element 11 through the optical system 10. 
Image pickup signals output from the image pickup 
element 11 are successively written into the image 
memory 2 through an analog/digital (A/D) converter 
12. The reference numeral 13 denotes a lens drive sys- 
tem for effecting the focus adjustment of the optical 
system 10. The CPU 1 supplies a lens portion control 
signal SI to the lens drive system 13, whereby the focus 
adjustment of the optical system 10 is effected. 

Description will now be made of the procedure when 
in the image processing apparatus of the present em- 
bodiment, the image data of the template which pro- 
vides the reference of focus detection is registered. 

The image of the sample 9 including a plurality of 
object patterns is first picked up by the image pickup 
element 11, and then image data output from the image 
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pickup element 11 is written into the image memory 2 
through the A/D converter 12, and the image data 
written into this image memory 2 is supplied to the 
image display 5 through the D/A converter 4. Thereby, 

5 as shown in FIG. 2, an image corresponding to data 14 
on the image memory 2 (i.e., the predetermined magni- 
fied real image of the sample 9) is displayed on the 
image display 5. At this time, the operator effects focus- 
ing through the CPU 1 and the lens drive system 13 so 

10 that for example, the image which is the template in the 
image being displayed on the image display 5 may be- 
come most clear-cut. 

Thereafter, the operator designates a rectangular 
window on the screen of the image display 5 by the use 

IS of the coordinates input unit 8, whereupon the image 
data on the image memory 2 which corresponds to the 
image in this window is taken out by the CPU 1, and the 
thus taken-out image data is stored (i.e., registered) as 
the image data of the in-focus state of the template in the 

20 external store 3. As the template, various kinds of tem- 
plates are registered in conformity with observation 
objects. 

At this time, as shown in FIG. 2, two kinds of win- 
dows, Le., a relatively large window Wl and a rela- 

25 tively small window W2, may be prepared, and the 
image data of a rough template Tl and the image data of 
a fine template T2 may be cut out by the use of these 
two kinds of windows Wl and W2, and the thus cut-out 
image data of the two kinds of templates may be regis- 

30 tered in the external store 3. The rough template Tl is 
used to effect focusing at a high speed, and the fine 
template T2 is used to effect focusing highly accurately. 

The operation when auto focusing is effected by the 
use of the registered image data of the templates will 

35 now be described with reference to the flow chart of 
FIG. 4. First, at the step 101 of FIG. 4, the operator 
downloads the image data of the template preselected 
from within the external store 3 into the CPU 1. There- 
after, at a step 102, the operator sets the sample 9 includ- 

40 ing a plurality of object patterns in front of the optical 
system 10, whereupon the image data of the sample 9 is 
written into the image memory 2. Then, the search of 
positions in the entire image field of the sample at which 
object patterns having a high correlation with the refer- 

45 ence pattern exist (hereinafter referred to as the candi- 
date points) is effected by the CPU 1. Specifically, the 
CPU 1 calculates the distribution of the degree of coin- 
cidence with the image data of the templates in all the 
image data on the image memory 2 (the degree of coin- 

50 cidence as it is made into a numerical value is referred to 
as the "correlation value of pattern"), and extracts a 
portion in which the correlation value of pattern is 
greater than a threshold value as a candidate point. 
When no candidate point is detected, the operation of 

55 the CPU 1 shifts from a step 103 to a step 104, where the 
CPU 1 transmits a control signal for lens portion to the 
lens drive system 13. In conformity therewith, the focus 
adjustment of the optical system 10 is effected, and the 
real image of the sample 9 as its focus state has changed 

60 is formed on the image pickup element 11, and at the 
step 102, the search of candidate points is effected again 
with this image as an object. 

When candidate points are detected, the operation of 
the CPU 1 shifts from the step 103 to a step 105, where 

65 the search field of the data on the image memory 2 is 
fixed around the already extracted candidate points. At 
the subsequent step 106, the CPU 1 transmits the con- 
trol signal SI for lens portion to the lens drive system 
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13, whereby the image data of the sample 9 as its focus tive patterns, and then the image data are registered, 
state has changed is written into the image memory 2. That is, it being understood that two patterns PA and 
At a step 107, the CPU 1 calculates the correlation PB to be registered as templates exist within the obser- 
value of pattern with the image data of the templates, vation screen 15 of FIG. 6A, it is to be understood that 
only regarding the surroundings of those candidate 5 the heights of these patterns PA and PB on the sample 
points, and records the correlation value of pattern are Zl and 22 (Z1>Z2), respectively. In this case, to 
correspondingly to the focal position. Generally, when resister the image data of the pattern PA, the pattern 
the correlation value of pattern is recorded correspond- PA on the observation screen 15 is surrounded by a 
ingly to the focal position, there is obtained a "focal window W3 with the focus adjusted to the height Zl on 
point evaluation function" which assumes a high peak at 10 the sample. The image data of the area surrounded by 
the focal point as shown in FIG. 3. this window W3 (FIG. 6B) is registered as the image 

Until the peak of the focal point evaluation function is data of a first template T3. Subsequently, the pattern PB 
detected at all candidate points, the operation of the on the observation screen 15 is surrounded by a window 
CPU 1 returns from a step 108 to the step 106 and the W4 with the focus adjusted to the height Z2 on the 
calculation of the correlation value of pattern around 15 sample. The image data of the area surrounded by this 
the candidate points is repetitively executed. When the window W4 (FIG. 6C) is registered as the image data of 
peak of the focal point evaluation function is detected at a second template T4. It is to be understood that after 
all candidate points, the operation of the CPU 1 shifts the image data of FIGS. 6B and 6C are registered as the 
from the step 108 to a step 109, where the focus state is image data of the templates, the operation shown in 
finally returned to a focal position at which the correla- 20 FIG. 4 is executed with the observation screen 15 of 
tion value of pattern becomes highest. This is executed FIG. 6A as the object of focus detection. In this case, if 
by the CPU 1 controlling the lens drive system 13 the image data of FIG. €B is used, there is obtained a 
through the control signal SI for lens portion, by the focal point evaluation function indicated by the func- 
use, for example, of the popular signal ascending tion ISA of FIG. 7, and by adjusting the focal position 
method. 25 to the peak of the function 18A, the focus can be ad- 

By way of example, assume that three candidate justed to the pattern PA of FIG. 6A. On the other hand, 
points 16A-H6C are first obtained in the observation if the image data of FIG. 6C is used, there is obtained a 
image field 15 by the search of candidate points, as focal point evaluation function indicated by the func- 
shown in FIG. 5A. The focal point evaluation functions tion 18B of FIG. 7, and by adjusting the focal position 
at these candidate points 16A-16C become the func- 30 to the peak of the function 18B, the focus can be ad- 
tions 17A-17C, respectively, of FIG. 5B, and it is the justed to the pattern PB of FIG. 6A. Thereby, focus 
focal point evaluation function 17A at the candidate detection can be effected accurately irrespective of the 
point 145A that is highest in the peak of the correlation heights of the templates. 

value I of pattern. From this, it is seen that a pattern Further, in the example shown, for instance, in FIG. 
identical or similar to the template exists only at the 35 5A, the candidate point 16A is finally regarded as a 
candidate point 16A. The focal position is then adjusted pattern most similar to the template and the automatic 
to the peak point of the function 17A, whereby focus focusing operation is executed so that the image of the 
detection is effected for the pattern identical or similar candidate point 16A may be most clear-cut, but design 
to the template. may be made such that the automatic aligning operation 

Also, the secondary peaks of the correlation values of 40 is executed at the same time. In this automatic aligning 
pattern obtained at the other candidate points 16B and operation, the position of the sample 9 of FIG. 1, the 
16C than the correct candidate point 16A assume levels direction of the optical system 10 or the position of the 
sufficiently lower than the peak signal intensity ob- image pickup element 11 is adjusted, whereby adjust- 
tained at the correct candidate point. So, as shown in ment is effected so that patterns identical or similar to 
FIG. 5B, a threshold value Iomay be set for the correla- 45 the templates may always come to predetermined posi- 
tion value I of pattern, and during the judgment of the tions within the observation screen 15. More specifi- 
detection of the candidate points at the step 103 of FIG. cally when, for example, in FIG. 5 A the candidate point 
4, only that portion of the correlation value I of pattern 16A is regarded as most similar to the template, the 
which exceeds the threshold value Io may be regarded amount of deviation between the position of that candi- 
as a candidate point. By such narrowing of the candi- 50 date point 16A on the observation screen 15 and a pre- 
date point, focus detection can be improved. determined position is detected. The CPU 1 of FIG. 1 

As described above, according to the present embodi- modifies the relative positional relation between the real 
ment, the image data of the template is registered in image of the sample 9 and the image pickup element 11 
advance, whereafter the correlation value of pattern of so as to negate this amount of deviation. When the 
the object pattern on the observation screen and the 55 in-focus state is varied by this automatic aligning opera- 
templates is found while the optical system 10 is moved tion, the automatic focusing operation is also executed 
in the direction of the optical axis to thereby vary the together, 
relative positional relation among the image pickup 

device 11, the optical system 10 and the sample 9, and a Second Embodiment 

relative positional relation for which the correlation 60 A second embodiment of the present invention will 
value of pattern becomes highest is regarded as the now be described. The basic construction of the hard- 
in-focus state. Accordingly, at whatever position on the ware of the present embodiment is the same as the con- 
image field an object pattern identical or similar to that struction (FIG. 1) of the first embodiment, and the 
template may exist, focusing can always be effected on image data of a template is prepared in a procedure 
that pattern. 65 similar to that in the first embodiment. In the present 

Where the shapes of patterns which are the objects to embodiment, however, the normalization of gradation 
be registered as templates have level differences as is effected when the image data of a template is regis- 
shown in FIG. 6A, focusing is effected on the respec- tered 
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Assuming that the template to be registered is a pat- template of which the gradation has been normalized 

tern in the window W5 of FIG. 8A f it is to be under- and therefore, the focal position difference to the focal 
stood that the distribution of the gradation of all pixels point can be found quickly from the correlation value of 
in the window W5 at the stage of raw data is repre- pattern obtained, and focus detection can be effected at 

sented by the solid line 19 A of FIG.8B. In the distribu- 5 a higher speed. Further, not only the image data of the 

tion indicated by this solid line 19A, there are many template but also the image data which is the observa- 

pixels of levels in the vicinity of the center. So, in the tion object to be searched may be subjected to the nor- 

present embodiment, the ma x imum value and minimum malizing process. Again in this case, focus detection will 

value of the gradation in the distribution indicated by be sped up. 

the solid line 19A are detected and the distribution of 10 Besides the normalization of gradation, for example, a 
the gradation is adjusted so that the actually measured pre-process of binarizing image data will now be con- 
maximum value and minim u m value may become the sidered. When the template which is the object of this 
theoretical maximum value and minimu m value, respec- binarization is a pattern within an area 21 shown in FIG. 
tively, of the gradation. Thereby, as indicated by the 10A, it is to be understood that the distributed state of 
broken line 19B of FIG. 8B, there is registered such 15 the gradation of pixels within this area 21 is represented 
image data that the width of the distribution of the by a solid line indicated in FIG. 10B. As threshold 
gradation has a m a xim um value. This means that the values for binarizing this image data, there are conceiv- 
image data of the template is registered after normaliza- able, for example, three kinds of threshold values thl, 
tion so that light and shade may become clearer. th2 and th3 (thl<th2<th3) shown in FIG. 10B. If a 
f Subsequently, by the use of the image data of the 20 focal point evaluation function is to be found for an 
template of which the gradation has been normalized, identical or similar pattern by the use of the image data 
the search of candidate points is effected for a sample in of the template binarized by the threshold value th2, 
the same manner as in the first embodiment. Again in there is obtained the function 22 of FIG. 10C, and if a 
this case, the correl ation value of the focal po int evalua- focal point evaluation function is to be found for an 
tionfimctio n_of a candlaate point at which th ehighest 25 identical or similar pattern by the use of the image data 
correlaTion^ajue^ been obtam eais 'eva lu- of the template binarized by the threshold value thl or 
a tea as an absolute value. ljrirherem. referr ed,to.a s th3, there is obtained the function 23 of FIG. IOC. The 
"normalization correlation search" thatJhe_search^of function 22 assumes a great peak at the focal point, 
candidate pojntsJs^ ^ed^by.the,use_of-the ima gejdata while the function 23 does not assume a great peak even 
oftnTthus noraaliz^ .template.-Norraaliza^ 30 at the focal point and thus, there is the possibility of 
Hon search is effected on a pattern identical or similar to accurate focus detection being not accomplished. This 
that template, whereby a normalization correlation fr^panq that if the image data is simply binarized, the 
search model curve 20 as shown in FIG. 9 is obtained as shape of the focal point evaluation function will become 
the model curve of the focal point evaluation function. irregular by the threshold value of the binarization. In 
In this model curve 20, the correlation value of the peak 35 contrast, according to the process of the present em- 
at the focal point is 100. Also, it has been found that bodiment for normalizing the gradation, there can al- 
when normalization correlation search is effected, the ways be obtained substantially the same focal point 
focal point evaluation function obtained always assumes evaluation function and focus detection can always be 
a curve substantially equal to the model curve 20 with effected accurately and rapidly, 
the focal point as the center. 40 

So, by the characteristic of the normalization correla- Third Embodiment 

tion search model curve 20 being stored in advance in a In a third embodiment, assuming that in the same 

memory, the difference to the focal point can be esti- image pickup field, there exist a plurality (N) of object 

mated quickly from the correlation value of pattern patterns differing in focal positions, there is handled a 

obtained. When for example, CI is obtained as a correla- 45 case where focus detection is continuously and succes- 

tion value of pattern by normalization correlation sively effected for those object patterns differing in 

search, a focal position at which the correlation value is focal position. The basic construction of the hardware 

CI in the normalization correlation search model curve of this third embodiment also is the same as the con- 

20, as shown in FIG. 9, is found. Since the focal point is struction (FIG. 1) of the first embodiment, and the 

already known, the difference Sz from that focal posi- 50 image data of templates are prepared in a similar proce- 

tion to the focal point can be immediately found, and dure. 

this difference Sz is an estimated value to the actual First, as shown in FIG. 11, for example, three pat- 
focal point Accordingly, by the CPU 1 of FIG. 1 caus- terns P24A, P24B and P24C which provide templates 
ing the lens drive system 13 by the use of the control are subjected to focusing in the observation screen 15 
signal SI for lens portion to move the focal position by 55 for a sample of which the focus detection is to be ef- 
the difference 8z, the focal position is immediately fected and are surrounded by windows W24A, W24B 
moved to the vicinity of the peak of the actual focal and W24C, respectively, whereby the image data of 
point evaluation function. Thereafter, focusing is ef- three templates T24A, T24B and T24C are extracted, 
fected near that peak by the ascending method or the These image data are stored in the external store 3 of 
like, whereby accurate focus detection is finally ef- 60 FIG. 1. Then, a sample of which the image is to be 
fected. At this time, as shown in FIG. 9, two focal actually picked up is introduced into the field of view, 
positions at which the correlation value is e.g. CI exist and the image data of a plurality of templates for focus- 
rightwardly and leftwardly of the focal point, but the ing already stored are designated and down-loaded into 
current focal position can be easily discriminated by the CPU 1. 

whether the correlation value becomes high or low 65 Subsequently, candidate points at which the correla- 

when the focal position is shifted a little. tion value of pattern is high are searched in each tem- 

In the present embodiment, normalization correlation plate on the observation screen while the template to be 

search is effected by the use of the image data of the used at the current focal position is replaced with an- 
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other. At this time, depending on templates, it some- described normalization correlation process or the filter 
times happens that on the observation screen, all the process of emphasizing the edge portions of patterns is 
correlation values of pattern are below a predetermined conceivable as a pre-process for the templates and pat- 
candidate point detection level. In such case, the focal terns to be searched, and by carrying out these pre- 
position is detected, and the detected focal position is 5 processes, stable focus detection can be executed even 
changed at a predetermined step width and candidate for lmear images of which only the outline is clear-cut, 
points at which the correlation value of pattern is high or ^ e ^ 

are searched again, whereby candidate ^po-inte are de- where for example, the object of focus detection is an 
tected m all templates. On the other hand, for templates oc cw,™ £ ct P + : , ."° f" 

in which candidate points have been decided, the focal 10 ^LZ^ZT \ 5* °7^T^T^t 
position is changed at the step width thereof and the 10 % ^^a ° f F ??? A * ^ 

focal point evaluation function is measured at each *™** 26 * £*™ ^^mlaton Wiethe template 
candidate point and finally, as shown in FIG. 12, there of ? fFlG ' } 3B ls J^ h m an area 27 ' correla - 

are obtained focal point evaluation functions 25A-25C ^ the template of the lme of mtersection of FIG. 

in which focal points are Z1-Z3 for the three templates 15 130 18 "S* m 311 area 28 » md ^ correlation with the 
T24A, T24B and T24C, respectively. In this case, it is to template of the short straight line of FIG. 13D is high in 
be understood that Z1-Z2=8Z1, Z2-Z3= -8Z2 and areas 29A 311(1 29B - Accordingly, as in the first embodi- 
Z3-Z1=6Z3 are established. ment ; the image data of the standard template of FIG. 

Then, the inter-peak focal position difference of the * 3A ^ ^t down-loaded into the CPU 1, whereafter the 
focal point evaluation function between any two tem- 2 q °f ^ s^pte of FIG. 14 is picked up and the 

plates is found and a focal position relative relation table distribution of the degree of correlation of pattern in the 
as shown in Table 1 below is prepared. whole image field is calculated. Thereby, the areas 26A, 

TABLE 1 etc * m detected 38 candidate points at which the 

correlation value of pattern is high. 
^ Subsequently, with the marginal area of the candidate 
points as the search area, any variation in the degree of 
correlation of pattern is monitored to thereby obtain 
focal point evaluation functions while the control signal 
, , SI for lens portion is sent from the CPU 1 of FIG. 1 to 

£« te J£f ' ? ^P^' * e Pineal values ^ lens ^ system a ^ focal position ^ driven 

"T^ VSl . C h ° m i ntal ™* * e . tem " a position indicative of the peak value of one of 

plate T24A mean that when the ^template TMA « the ^ focal p^t evaluation functions which exhibits the 

T24C differ by -6Z1 and 8Z3 respectively. cuted Like wise, focus detection can also be effected 

Descnbmg thew^^ accurately by the use of the standard templates of 

tive relation table, it is to be understood that focus de- pjqs 13A-13D H 

tection is continuously effected in the pre-designated tw^i—i,, \ ... . . , . - 

orderoftem P lates.Inthiscase,themovementsequence J2?£tSr ^ ^ ^1°/ 

of the focal position is prepared with reference to the f™^ f ™ age ^ 3 efficiently small level dif- 

focal position relative relation table and during the J erence ,°I V**™ relatlve t0 the ^ of 

actual movement, the focal position is continuously * focus of the image pickup system, like a microscopic 

adjusted in accordance with that movement sequence, T f ' [ ocv ? dQt **?* can iL be effected sufficiently accu- 

whereby focus detection more efficient than individu- rately by techm( l ue ^ m ^ fourth embodi- 

ally effected focus detection becomes possible. Specifi- m t?£. 

cally, to effect focus detection to the template T24A, .where any images having various distances from a 

for example, after the template T24B, focus detection to 45 ^deo camera to the object of image pickup are picked 

the template T24B is completed, whereafter the focal U P' for exam P ,e > b Y ^ video camera, the search area 

position is shifted upwardly by 8Z1 in accordance with mav be restricted to a predetermined area in the obser- 

Table I, whereby focus detection can be effected very vati on screen at the stage of the search of candidate 

quickly. points which determine an area for effecting focus de- 

50 tection accurately. In this manner, the degree of corre- 

Fourth Embodiment lation of pattern is calculated only for a pattern in the 

The first to third embodiments handle a case where predetermined area to find a focal point evaluation 

focus detection is effected to a specific pattern, while a function and in conformity with the result of this, focus 

fourth embodiment handles a case where focus detec- detection is executed, whereby focus detection can be 

tion is effected to an unspecific pattern. Again in the 55 effected at a higher speed, 

present embodiment, the basic construction of the hard- Even when as in the above-described embodiments, 

ware is the same as the construction (FIG. 1) of the first one of toe image pickup device, the optical system and 

embodiment. the observation object is moved in the direction of the 

Generally, even when a sample of any shape is being optical axis of the optical system, the relative positional 

observed, right-angled corner portions, a portion of an 60 relation among the image pickup device, the optical 

arc, lines of intersection and a short straight line exist in system and the observation object becomes varied. This 

the picked-up image. Accordingly, the image data of also holds true when a part of the optical system is 

patterns as shown in FIGS. 13A-13D are registered in moved 

advance as the image data of standard templates in the Thus, the present invention is not restricted to the 

external store 3 of FIG. 1. Particularly, where the pat- 65 above-described embodiments, but can assume various 

tern which provides the reference for focus detection is constructions without departing from the basic princi- 

not restricted, focus detection is executed by the use of pies of the invention, 

such standard templates. In this case, the above- What is claimed is: 
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1. A focus detecting method comprising the steps of: 

(a) projecting a real image of an observation object 
including a plurality of object patterns onto an 
image pickup device through an optical system, 
and producing image data from an output of said 
image pickup device; 

(b) calculating correlation values of the image data of 
each of said plurality of object patterns and image 
data of a prestored reference pattern while varying 
a relative positional relation among said image 
pickup device, said optical system and said obser- 
vation object in a direction of an optical axis of said 
optical system; and 

(c) judging the relative positional relation among said 
image pickup device, said optical system and said 
observation object which gives a maximu m corre- 
lation value as an in-focus state. 

2. A focus detecting method according to claim 1, 
wherein said calculating step includes: 

finding the correlation value of the image data of 20 
each of said plurality of object patterns and the 
image data of said reference pattern when said 
image pickup device, said optical system and said 
observation object are in a certain relative posi- 
tional relation, selecting an object pattern for 25 
which said correlation value is a predetermined 
value or greater from among said plurality of ob- 
ject patterns, 

shifting the relative positional relation among said 
image pickup device, said optical system and said 30 
observation object from said certain relative posi- 
tional relation to a relative positional relation dif- 
fering therefrom, and finding the correlation value 
of the image data of said selected object pattern 
and the image data of said reference pattern with 35 
said image pickup device, said optical system and 
said observation object in said differing positional 
relation. 

3. A focus detecting method according to claim 1, 
further including the steps of: 40 

displaying the real image of said observation object 
on a screen based on the image data produced from 
the output of said image pickup device; and 

relatively moving at least one of the real image of said 
observation object and said image pickup device in 45 
a direction perpendicular to said optical axis so that 
one of said plurality of object patterns for which 
the correlation with said reference pattern is high- 
est is brought to a predetermined position on said 
screen. 50 

4. A focus detecting method comprising the steps of: 

(a) projecting a real image of an observation object 
including a plurality of object patterns onto an 
image pickup device through an optical system, 
and producing image data from an output of said 55 
image pickup device; 

(b) normalizing the image data of each of said plural- 
ity of object patterns so that a difference between a 
maximum value and a minimum value of gradation 
of pixels represented by that image data becomes a 60 
predetermined value; 

(c) finding correlation values of the normalized image 
data of each of said object patterns and image data 
of a prestored reference pattern while varying a 
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judging the relative positional relation among said 
image pickup device, said optical system and said 
observation object which gives a maximum corre- 
lation value as an in-focus state. 

5. A focus detecting method according to claim 4, 
wherein said calculating step includes: 

finding the correlation value of the normalized image 
data of each of said object patterns and the image 
data of said reference pattern when said image 
pickup device, said optical system and said obser- 
vation object are in a certain positional relation, 
selecting an object pattern for which said correla- 
tion value is a predetermined value or greater from 
among said plurality of object patterns, 

shifting the relative positional relation among said 
image pickup device, said optical system and said 
observation object from said certain relative posi- 
tional relation to a relative positional relation dif- 
fering therefrom, and finding the correlation value 
of the image data of said selected object pattern 
and the image data of said reference pattern with 
said image pickup device, said optical system and 
said observation object in said differing positional 
relation. 

6. A focus detecting method according to claim 4, 
further including the steps of: 

displaying the real image of said observation object 
on a screen based on the image data produced from 
the output of said image pickup device; and 

relatively moving at least one of the real image of said 
observation object and said image pickup device in 
a direction perpendicular to said optical axis so that 
one of said plurality of object patterns for which 
the correlation with said reference pattern is high- 
est is brought to a predetermined position on said 
screen. 

7. A focus detecting apparatus including: 

an image pickup device for picking up a real image of 
an observation object including a plurality of ob- 
ject patterns through an optical system; 

drive means for driving at least one of said image 
pickup device, said optical system and said obser- 
vation object in a direction of an optical axis of said 
optical system; 

first memory means for storing therein image data of 
said object patterns obtained from an output of said 
image pickup device; 

second memory means for prestoring image of data of 
a reference pattern therein; 

calculating means for calculating correlation values 
of the image data in said first memory means and 
the image data in said second memory means while 
controlling said drive means to vary a relative 
positional relation among said image pickup de- 
vice, said optical system and said observation ob- 
ject; and 

judging means for judging the relative positional 
relation among said image pickup device, said opti- 
cal system and said observation object which gives 
a maximum correlation value as an in-focus state. 

8. A focus detecting apparatus according to claim 7, 
further including normalizing means for normalizing 
the image data of each of said object patterns so that a 



difference between a maximum value and a minimum 
relative positional relation among said image 65 value of gradation of pixels represented by that image 
pickup device, said optical system and said obser- data becomes a predetennined value, and wherein said 
vation object in a direction of an optical axis of said first memory means stores therein the normalized image 
optical system; and data of the object patterns. 



09/09/2004, EAST Version: 1.4.1 



15 



5,369,430 



9. A focus detecting apparatus according to claim 7, 
further including: 

display means for displaying the real image of said 
observation object on a screen based on the image 
data obtained from the output of said image pickup 
device; and 

moving means for relatively moving at least one of 
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the real image of said observation object and said 
image pickup device in a direction perpendicular to 
said optical axis so that one of said plurality of 
object patterns for which the correlation with said 
reference pattern is highest can be brought to a 
predetermined position on said screen. 
***** 



10 



15 



20 



25 



30 



35 



40 



45 



50 



55 



60 



65 



09/09/2004, EAST Version: 1.4.1 



